Introduction of field API interface in ectrans #314

dhaumont · 2025-09-26T08:39:16Z

Summary

This PR introduces a field API interface to ectrans. It is based on top of dump-checksums (PR #287)

Description of changes

The new field API interface is implemented in src/field_api:
- src/field_api/trans/inv_trans_field_api.F90: inverse transform (calling inv_trans internally)
- src/field_api/trans/dir_trans_field_api.F90: direct transform (calling dir_trans internally)
- src/field_api/field_api_ectrans_mod.F90: helper functions
Optional compilation triggered by a new USE_FIELD_API configuration option. Adds a new dependency to field API.
Single and double precision for CPU and GPU
Accessible in ectrans-benchmark via a new option --field-api
- src/programs/util/ectrans_field_api_helper.F90 contains the field API implementation
In the GPU version of ectrans-benchmark , setting llacc = .true. will force the intermediate fields on GPU. This requires additional memory copies host <-> device before and after the regular ectrans calls.
New unit tests have been added to test --field-api (CPU and GPU)
compare_checksums.py updated to compare the checksums with field-api with the regular version
nvhpc22.11 on CPU: bit reproducibility between the field API and the regular version has been partially adressed by clamping small values in spectral space after dir_trans in ectrans benchmark (2a7aa87). This only applies to double precision (single precision is only bit reproducible during the first time step and slightly different after)

Testing and validation

ATOS:
- Intel 2021.4.0 - CPU: bit reproducible
- nvhpc 22.11 - CPU: dp : bit reproducible, sp: not reproducible
- nvhpc 22.11 on GPU: bit reproducible
Ubuntu laptop
*Gfortran 13.3.0 - CPU : bit reproducible

Known limitations

no LAM version
no openMP version ( OpenACC only)
The GPU version with llacc = .true. requires memory transfers between CPU and GPU, that should be avoided
for nvhpc 22.11, the CPU version in sp is not bit reproducible ( first time step is bit reproducible, tiny differences after first iteration)

Initial pull request

dhaumont#1
Contains some comments from @samhatfield that needs to be integrated

Prototype in ARPEGE

See for instance this prototype of transinv_mdl_field_api.F90: https://github.com/dhaumont/IAL/blob/50t1_field_api_ectrans/arpifs/transform/transinv_mdl_field_api.F90
This version can be compiled using this bundle:
50t1_ectrans_field_api_interface

Without this a test with an incorrect argument still passes.

…into dump-checksums

samhatfield · 2026-01-08T12:53:36Z

src/field_api/inv_trans_field_api.F90

+INTEGER(KIND=JPIM)          :: IFLDXGUV
+INTEGER(KIND=JPIM)          :: IFLDXLUV


Suggested change

INTEGER(KIND=JPIM) :: IFLDXGUV

INTEGER(KIND=JPIM) :: IFLDXLUV

These aren't used

done in ecbf2d2

samhatfield · 2026-01-08T12:53:51Z

src/field_api/inv_trans_field_api.F90

+IFLDXGUV= 0
+IFLDXLUV= 0


Same as above

good catch. they will go away

done in ecbf2d2

samhatfield · 2026-01-08T12:54:41Z

src/field_api/inv_trans_field_api.F90

+!     ------------------------------------------------------------------
+IF (LHOOK) CALL DR_HOOK('INV_TRANS_FIELD_API',0,ZHOOK_HANDLE)
+
+ISPUV = 0


ISPUV also doesn't seem to be used - can you check?

correct. I am removing it.

done in 7cc7912

src/field_api/inv_trans_field_api.F90

samhatfield · 2026-01-08T13:17:15Z

src/field_api/ectrans_field_api_mod.F90

+  ! Spectral field view
+  REAL(KIND=JPRB),POINTER :: P(:)
+  INTEGER               :: IVSET
+  CHARACTER(LEN=12)     :: NAME


I couldn't find a case where NAME is used. Could you check this is still required?

Names are really useful for debugging, I would prefer to keep them

samhatfield · 2026-01-08T13:17:20Z

src/field_api/ectrans_field_api_mod.F90

+! Grid point field view
+  REAL(KIND=JPRB),POINTER :: P(:,:)
+  INTEGER               :: IVSET
+  CHARACTER(LEN=12)     :: NAME


wdeconinck · 2026-01-08T13:54:59Z

I took the liberty to commit directly in your branch @dhaumont, I hope you don't mind.
I fixed the CI by disabling the tests that require field_api when the feature FIELD_API is not enabled.

wdeconinck · 2026-01-08T14:11:23Z

It is not possible that a downstream package would be linking to multiple versions of ectrans_field_api (i.e. ectrans_field_api_cpu_sp, ectrans_field_api_cpu_dp, ectrans_field_api_gpu_sp, ectrans_field_api_gpu_dp).
This is because you would have each ectrans_field_api_cpu_dp and ectrans_field_api_cpu_sp etc, each using FIELD_MODULE from field_api but this module and its types/symbols are not uniquely defined (once for field_api_sp and once for field_dp).
This is not a problem because its use is intended for IAL/IFS, which would link to only one of these versions.

However this means that we don't need to go through the same procedure as done with trans to define unique symbols, as the uniqueness is not important. For trans this was important because we want to be able to make use of trans through downstream libraries (like Atlas) that may make a runtime choice on precision and cpu/gpu execution.

If you don't mind I will refactor this a bit, reducing the complexity also with the includes which would not be required any longer.

dhaumont · 2026-01-08T15:28:58Z

I took the liberty to commit directly in your branch @dhaumont, I hope you don't mind. I fixed the CI by disabling the tests that require field_api when the feature FIELD_API is not enabled.

So the field api interface is not compiled nor executed in the CI ????
In my opinion, what is not tested is just dead code.

dhaumont · 2026-01-08T15:37:32Z

It is not possible that a downstream package would be linking to multiple versions of ectrans_field_api (i.e. ectrans_field_api_cpu_sp, ectrans_field_api_cpu_dp, ectrans_field_api_gpu_sp, ectrans_field_api_gpu_dp). This is because you would have each ectrans_field_api_cpu_dp and ectrans_field_api_cpu_sp etc, each using FIELD_MODULE from field_api but this module and its types/symbols are not uniquely defined (once for field_api_sp and once for field_dp). This is not a problem because its use is intended for IAL/IFS, which would link to only one of these versions.

However this means that we don't need to go through the same procedure as done with trans to define unique symbols, as the uniqueness is not important. For trans this was important because we want to be able to make use of trans through downstream libraries (like Atlas) that may make a runtime choice on precision and cpu/gpu execution.

If you don't mind I will refactor this a bit, reducing the complexity also with the includes which would not be required any longer.

One of the main requirement of this interface is the ability to dynamically choose betweeen CPU and GPU execution: you want to be able to run the spectral transform on CPU or on GPU with LDACC.
Is this compatible with what you suggest ?

I'm not sure what problem you are trying to solve exactly, is this the fact that you have to include an interface file ?

samhatfield · 2026-01-08T15:58:09Z

So the field api interface is not compiled nor executed in the CI ???? In my opinion, what is not tested is just dead code.

It's tested in the "HPC" suite (ECMWF HPC2020 and LUMI) but not the Github suite. So what we need is a commit like 88af96b but applied to .github/workflows/build.yml.

wdeconinck · 2026-01-08T16:04:43Z

So the field api interface is not compiled nor executed in the CI ???? In my opinion, what is not tested is just dead code.

The FIELD_API feature is optional, meaning that when FIELD_API is not enabled those particular tests should not be running, we can all agree on that. That is what I fixed.
Next, I definitely agree that the CI should be running also with FIELD_API feature enabled.

I forgot to mention that for the build-hpc workflow I also added the missing field_api dependency in commit ce5ae51 . It was not sufficient to set "-DENABLE_FIELD_API=ON", so hopefully the CI is going to test this at least on the HPCs now (lumi and ecmwf-atos).
Looking at the logs however it seems to ignore changes to the Github-Actions workflows for pull-requests with "secrets". I guess that is a "feature" of Github-Actions for security reasons. We would need to first change the workflow in the develop branch then to make sure this PR is testing FIELD_API changes.

dhaumont · 2026-01-08T16:17:08Z

So the field api interface is not compiled nor executed in the CI ???? In my opinion, what is not tested is just dead code.

The FIELD_API feature is optional, meaning that when FIELD_API is not enabled those particular tests should not be running, we can all agree on that. That is what I fixed. Next, I definitely agree that the CI should be running also with FIELD_API feature enabled.

I forgot to mention that for the build-hpc workflow I also added the missing field_api dependency in commit ce5ae51 . It was not sufficient to set "-DENABLE_FIELD_API=ON", so hopefully the CI is going to test this at least on the HPCs now (lumi and ecmwf-atos). Looking at the logs however it seems to ignore changes to the Github-Actions workflows for pull-requests with "secrets". I guess that is a "feature" of Github-Actions for security reasons. We would need to first change the workflow in the develop branch then to make sure this PR is testing FIELD_API changes.

OK, thanks a lot Willem.

wdeconinck · 2026-01-08T16:26:00Z

One of the main requirement of this interface is the ability to dynamically choose betweeen CPU and GPU execution: you want to be able to run the spectral transform on CPU or on GPU with LDACC. Is this compatible with what you suggest ?

I'm not sure what problem you are trying to solve exactly, is this the fact that you have to include an interface file ?

I think the simplicification will still allow us to develop later on the ability to choose between CPU and GPU backends, but not between single and double precision, due to linking to two field_api libraries that are incompatible with each other, i.e. the meanings of FIELD_1RB, FIELD_2RB etc.

What will be possible in the future is to have two libraries : ectrans_field_api_${prec} (${prec}=(sp,dp)) which will link with both ectrans_cpu_${prec} and ectrans_gpu_${prec}.
Based on the LDACC flag you would then need to call invtrans_cpu_${prec} or invtrans_gpu_${prec} within invtrans_field_api(KRESOL=NRESOL, ..., LDACC=...)

It is however not yet clear to me if NRESOL is in this case the same value for LDACC=.TRUE. or LDACC=.FALSE.
We are not there yet. Perhaps the GPU or CPU backend will be embedded in the KRESOL, so that we don't need the LDACC flag:

call invtrans_field_api(KRESOL=NRESOL_GPU, ...)
call invtrans_field_api(KRESOL=NRESOL_CPU, ...)

and then introspection on the KRESOL parameter will need to be used instead of the LDACC flag.

What is clear to me however is that you also did not intend to create these symbols: invtrans_field_api_(cpu,gpu)_(sp,dp).

wdeconinck · 2026-01-08T16:55:28Z

See dhaumont#4 for the proposed changes to remove the unique symbol handling with the sedrenames etc.

Remove unique symbols for ectrans_field_api

…is now useless

dhaumont · 2026-01-09T15:27:51Z

@samhatfield @wdeconinck After our discussion about different passes in LS and LG that were not needed anymore, I simplified the code by making the rguments optional. I also removed the passes.

c7af7ea

IMPORTANT: I just found a crash in the unit test due to this commit. I'm inverstigating

samhatfield and others added 30 commits July 3, 2025 16:19

Exit with error when incorrect argument applied

5471960

Without this a test with an incorrect argument still passes.

Fix typo: baseargs -> base_args

9b1c055

Make sure base_args is expanded properly

07e1b4f

Initial implementation

653cecc

Implement dump-checksums

33cf796

typo

de15138

dump with gather (not working)

b61e65a

fix crashes

b3f485d

small refactoring

e24f2a9

typo

93377c3

add checksums for lam

f37bb3e

Merge remote-tracking branch 'ecmwf/develop' into dump-crcs

276bf7e

change filename

c79dd56

Add script to compare checksums

9e86a4a

python3

5a95205

Add unit test for checksum

21db9d2

Merge branch 'dump-checksums' of https://github.com/dhaumont/ectrans …

a13d6c8

…into dump-checksums

rename txt in checksums

f6a97e6

remove initialization of arrays before inv_trans

17731e8

rename checksums

327c5e1

adding underscore in checksums

b11f38e

remove typo correction after fix in another branch

881d086

Merge branch 'develop' into dump-checksums

c17ab29

Change color enum

35a6865

Split long function calls and reorder arguments

a1795a4

fix naming of checksums files

6c4c9e3

Fix indentation

ec38922

Fix out-of-bound access in Nvidia debug mode

67cf8ce

The test compare_checksum will trigger other tests

06d62a8

Indent if

4850f25

Add field_api as dependency in build-hpc CI

ce5ae51

wdeconinck added the approved-for-ci label Jan 8, 2026

samhatfield reviewed Jan 8, 2026

View reviewed changes

src/field_api/inv_trans_field_api.F90 Show resolved Hide resolved

samhatfield reviewed Jan 8, 2026

View reviewed changes

Remove unique symbols for ectrans_field_api

9bf2044

wdeconinck mentioned this pull request Jan 8, 2026

Remove unique symbols for ectrans_field_api dhaumont/ectrans#4

Merged

Merge pull request #4 from wdeconinck/field_api_interface

d4768aa

Remove unique symbols for ectrans_field_api

github-actions bot removed the approved-for-ci label Jan 9, 2026

wdeconinck added the approved-for-ci label Jan 9, 2026

dhaumont added 2 commits January 9, 2026 15:20

Unused variables

ecbf2d2

Add intent in and missing kind

059309a

github-actions bot removed the approved-for-ci label Jan 9, 2026

dhaumont added 2 commits January 9, 2026 15:46

Remove ISPUV

7cc7912

Refactor LS and LG with optional arguments and remove the JPASS that …

c7af7ea

…is now useless

Fix crash with Intel compiler

fd0ef29

dhaumont mentioned this pull request Jan 19, 2026

Factorize duplication between trltog and trgtol (cpu only) #317

Open

		INTEGER(KIND=JPIM) :: IFLDXGUV
		INTEGER(KIND=JPIM) :: IFLDXLUV

		IFLDXGUV= 0
		IFLDXLUV= 0

Introduction of field API interface in ectrans #314

Are you sure you want to change the base?

Introduction of field API interface in ectrans #314

Uh oh!

Conversation

dhaumont commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Description of changes

Testing and validation

Known limitations

Initial pull request

Prototype in ARPEGE

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wdeconinck commented Jan 8, 2026

Uh oh!

wdeconinck commented Jan 8, 2026

Uh oh!

dhaumont commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dhaumont commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samhatfield commented Jan 8, 2026

Uh oh!

wdeconinck commented Jan 8, 2026

Uh oh!

dhaumont commented Jan 8, 2026

Uh oh!

wdeconinck commented Jan 8, 2026

Uh oh!

wdeconinck commented Jan 8, 2026

Uh oh!

dhaumont commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dhaumont commented Sep 26, 2025 •

edited

Loading

dhaumont commented Jan 8, 2026 •

edited

Loading

dhaumont commented Jan 8, 2026 •

edited

Loading

dhaumont commented Jan 9, 2026 •

edited

Loading